Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation

摘要

Enabling robots to autonomously navigate complex environments is essentialfor real-world deployment. Prior methods approach this problem by having therobot maintain an internal map of the world, and then use a localization andplanning method to navigate through the internal map. However, these approachesoften include a variety of assumptions, are computationally intensive, and donot learn from failures. In contrast, learning-based methods improve as therobot acts in the environment, but are difficult to deploy in the real-worlddue to their high sample complexity. To address the need to learn complexpolicies with few samples, we propose a generalized computation graph thatsubsumes value-based model-free methods and model-based methods, with specificinstantiations interpolating between model-free and model-based. We theninstantiate this graph to form a navigation model that learns from raw imagesand is sample efficient. Our simulated car experiments explore the designdecisions of our navigation model, and show our approach outperformssingle-step and $N$-step double Q-learning. We also evaluate our approach on areal-world RC car and show it can learn to navigate through a complex indoorenvironment with a few hours of fully autonomous, self-supervised training.Videos of the experiments and code can be found at github.com/gkahn13/gcg